Reverse Code Engineering RCE CD +sandman 2000

home *** CD-ROM | disk | FTP | other *** search

/ Reverse Code Engineering RCE CD +sandman 2000 / ReverseCodeEngineeringRceCdsandman2000.iso / RCE / Softice Tuts / si-ug-chapter12.rtf < prev next >

Wrap

Text File | 2000-05-25 | 27KB | 679 lines

We will now discuss in a little more detail the struggle for existence. Charles Darwin SoftICE Tutorial Introduction Loading SoftICE Building the GDIDEMO Sample Application Loading the GDIDEMO Sample Application Controlling the SoftICE Screen Tracing and Stepping through Source Code Viewing Local Data Setting Point-and-Shoot Breakpoints Setting a One-Shot Breakpoint Setting a Sticky Breakpoint Using SoftICE Informational Commands Using Symbols and Symbol Tables Setting a Conditional Breakpoint Setting a BPX Breakpoint Editing a Breakpoint Setting a Read-Write Memory Breakpoint Introduction This tutorial gives you hands-on experience debugging a Windows application to teach you the fundamental steps for debugging applications and drivers. During this debugging session, you will learn how to do the following: * Load SoftICE * Build an application * Load the application source and symbol files * Trace and step through source code and assembly language * View local data and structures * Set point-and-shoot breakpoints * Use SoftICE informational commands to explore the state of the application * Work with symbols and symbol tables * Modify a breakpoint to use a conditional expression Each section in the tutorial builds upon the previous sections, so you should perform them in order. This tutorial uses the GDIDEMO application as its basis. GDIDEMO provides a demonstration of GDI functionality. GDIDEMO is located in the \EXAMPLES\GDIDEMO directory on your CDROM. GDIDEMO is also available under \mstools\samples\win32\GDIDEMO. If you use the GDIDEMO on the CDROM, copy it to your hard drive. You can substitute a different sample application or an application of your own design. The debugging principles and features of SoftICE used in this tutorial apply to most applications. Note: The examples is this tutorial are based on Windows NT. If you are using Windows 95, your output may vary slightly. Loading SoftICE If you are running SoftICE under Windows 95 or under Windows NT in Boot, System, or Automatic mode, SoftICE automatically loads when you start or reboot your PC. If you are running SoftICE in Manual Startup mode under Windows NT, SoftICE does not load automatically. To load SoftICE for Windows 95, enter the command WINICE. To load SoftICE for Windows NT, do one of the following: * Select START SOFTICE. * Enter the command: NET START NTICE Note: Once you load SoftICE, you cannot deactivate it until you reboot your PC. To verify that SoftICE is loaded, press the SoftICE hot key sequence Ctrl-D. The SoftICE screen should appear. To return to the Windows operating system, use the X (exit) or G (go to) command (F5). Building the GDIDEMO Sample Application The first step in preparing to debug a Windows application is to build it with debug information. The makefile for the sample application GDIDEMO is already set up for this purpose. To build the sample program, perform the following steps: 1. Open a DOS shell. 2. Change to the directory that contains the sample code. 3. Execute the NMAKE command: C:\MSTOOLS\SAMPLES\WIN32\GDIDEMO>NMAKE If GDIDEMO is located in another directory, change the path as appropriate. Loading the GDIDEMO Sample Application Loading an application entails creating a symbol file from the applicationÆs debug information and loading the symbol and source files into SoftICE. To Load the GDIDEMO application, perform the following steps: 1. Start Symbol Loader : The Symbol Loader window appears. 2. Either choose OPEN MODULE from the File menu or click the OPEN button : The Open window appears. 3. Locate GDIDEMO.EXE and click Open. 4. Either choose LOAD from the Module menu or click the LOAD button to load GDIDEMO. Symbol Loader translates the debug information into a .NMS symbol file, loads the symbol and source files, starts GDIDEMO, pops up the SoftICE screen, and displays the source code for the file GDIDEMO.C. Controlling the SoftICE Screen The SoftICE screen is your central location for viewing and debugging code. It provides up to seven windows and one help line to let you view and control various aspects of your debugging session. By default, it displays the following: Locals window: Displays and expand variables allocated on the stack. Code window: Displays source code or unassembled instructions. Command window: Enters user commands and display information. Help line: Provides information about SoftICE commands and shows the active address context. 1. Look at the contents of the Code window. Note that SoftICE is displaying the WinMain routine at line 34. By default, SoftICE creates a breakpoint and stops at the first main module it encounters when loading your application. 2. To see all the source files that SoftICE loaded, enter the FILE command with the wild card character: :FILE * SoftICE displays the source files for GDIDEMO: draw.c, maze.c, xform.c, poly.c, wininfo.c, dialog.c, init.c, bounce.c, and gdidemo.c. The Command window varies in size depending upon the number of lines used by open windows, so you might not see all these file names. To display the remaining file names, press any key. (Refer to Chapter 5: Navigating Through SoftICE on page 69 for information about resizing windows.) 3. Many SoftICE windows can be scrolled. If you have a mouse, you can click on the scroll arrows. If not, SoftICE provides key sequences that let you scroll specific windows. Try these methods for scrolling the Code window: Scroll the Code Window Key Sequence Mouse Action Scroll to the previous Click the innermost up page. PageUp scroll arrow Scroll to the next Click the innermost down page. PageDown scroll arrow Scroll to the previous Click the outermost up line. UpArrow scroll arrow Scroll to the next Click the outermost down line. DownArrow scroll arrow Scroll left one Click the left scroll character. Ctrl-LeftArrow arrow Scroll right one Click the right scroll character. Ctrl-RightArrow arrow 4. Enter the U command followed by EIP to disassemble the instructions for the current instruction pointer. :U EIP You can also use the . (dot) command to accomplish the same thing: :. Tracing and Stepping through Source Code The following steps show you how to use SoftICE to trace through source code: 1. Enter the T (trace) command or press the F8 key to trace one instruction. :T The F8 key is the default key for the T (trace) command. Execution proceeds to the next source line and highlights it. At this point, the following source line should be highlighted: if(!hPrevInst) 2. The Code window is currently displaying source code. However, it can also display disassembled code or mixed (both source and disassembled) code. To view mixed code, use the SRC command (F3). :SRC Note that each source line is followed by its assembler instructions. 3. Press F3 once to see disassembled code, then again to return to source code. 4. Enter the T command (F8) to trace one instruction. Execution proceeds until it reaches the line that executes the RegisterAppClass function. As demonstrated in these steps, the T command executes one source statement or assembly language instruction. You can also use the P command (F10) to execute one program step. Stepping differs from tracing in one crucial way. If you are stepping and the statement or instruction is a function call, control is not returned until the function call is complete. Hint: The T command does not trace into a function call if the source code is not available. A good example of this is Win32 API calls. To trace into a function call when source code is not available, use the SRC command (F3) to switch into mixed or assembly mode. Viewing Local Data The Locals window displays the current stack frame. In this case, it contains the local data for the WinMain function. The following steps illustrate how to use the Locals window: 1. Enter the T command to enter the RegisterAppClass function. The Locals window is now empty because local data is not yet allocated for the function. The RegisterAppClass function is implemented in the source file INIT.C. SoftICE displays the current source file in the upper left corner of the Code window. 2. Enter the T command again. The Locals window contains the parameter passed to the RegisterAppClass (hInstance) and a local structure wndClass. The structure tag wndClass is marked with a plus sign (+). This plus sign indicates that you can expand the structure to view its contents. Note: You can also expand character strings and arrays. 3. If you have a Pentium-class processor and a mouse, double-click the structure WNDCLASSA to expand it. To collapse the structure wndClass, double-click its contents. 4. To use the keyboard to expand the structure: press Alt-L to move the cursor to the Locals window, use the UpArrow or DownArrow to move the highlight bar to the structure, and press Enter. Press Enter again to collapse it. Setting Point-and-Shoot Breakpoints This section shows you how to set two handy types of point-and-shoot breakpoints: one-shot and sticky breakpoints. Setting a One-Shot Breakpoint The following steps demonstrate how to set a one-shot breakpoint. A one-shot breakpoint clears after the breakpoint is triggered. 1. To shift focus to the Code window, either use your mouse to click in the window or press Alt-C. If you wanted to shift focus back to the Command window you could press Alt-C again. Setting Point-and-Shoot Breakpoints 2. Either use the Down arrow key, the down scroll arrow, or the U command to place the cursor on line 61, the first call to the Win32 API function RegisterClass. If you use the U command, specify the source line 61 as follows: :U .61 SoftICE places source line 61 at the top of the Code window. 3. Use the HERE command (F7) to execute to line 61. The HERE command executes from the current instruction to the instruction that contains the cursor. The HERE command sets a one-shot breakpoint on the specified address or source line and continues execution until that breakpoint triggers. When the breakpoint is triggered, SoftICE automatically clears the breakpoint so that it does not trigger again. The following current source line should be highlighted: if(!RegisterClass(&wndClass)) Note: You can do the same thing by using the G (go) command and specifying the line number or address to which to execute: :G .61 Setting a Sticky Breakpoint The following steps demonstrate another type of point-and-shoot breakpoint: the sticky breakpoint, which does not clear until you explicitly clear it. The F9 key is the default key for the BPX command. 1. Find the next call to RegisterClass that appears on source line 74. With the cursor on line 74, enter the BPX command (F9) to set an execution breakpoint. The BPX command sets an execution breakpoint by inserting an INT3 instruction into the code. Note that the line is highlighted when you set a breakpoint. 2. Press the F9 key to clear the breakpoint. If you are using a Pentium-class processor and you have a mouse, you can double-click on a line in the Code window to set or clear a breakpoint. 3. Set a breakpoint on line 74, then use the G or X command (F5) to execute the instructions until the breakpoint triggers: :G When the INT3 instruction is executed, SoftICE pops up. Unlike the HERE command, which sets a one-shot breakpoint, the BPX command sets a sticky breakpoint. A sticky breakpoint remains until you clear it. 4. To view information about breakpoints that are currently set, use the BL command: :BL 00) BPX #0137:00402442 Note: The address you see might be different. From the output of the BL command, one breakpoint is set on code address 0x402442. This address equates to source line 74 in the current file INIT.C. 5. You can use the SoftICE expression evaluator to translate a line number into an address. To find the address for line 74, use the ? command: :? .74 void * = 0x00402442 6. The RegisterAppClass function has a relatively straightforward implementation, so it is unnecessary to trace every single source line. Use the P command with the RET parameter (F12) to return to the point where this function was called: :P RET The RET parameter to the P command causes SoftICE to execute instructions until the function call returns. Because RegisterAppClass was called from within WinMain, SoftICE pops up in WinMain on the statement after the RegisterAppClass function call. The following source line in WinMain should be highlighted: msg.wParam = 1; 7. Enter the BC command with the wild card parameter to clear all the breakpoints: BC * Using SoftICE Informational Commands SoftICE provides a wide variety of informational commands that detail the state of an application or the system. This section teaches you about two of them: H (help) and CLASS. 1. The H and Class commands work best when you have more room to display information, so use the WL command to close the Locals window. Closing this window automatically increases the size of the Command window. 2. The H command provides general help on all the SoftICE commands or detailed help on a specific command. To view detailed help about the CLASS command, enter CLASS as the parameter to the H command. :H CLASS Display window class information CLASS [-x] [process | thread | module | class-name] ex: CLASS USER The first line of help provides a description of the command. The second line is the detailed use, including any options and/or parameters the command accepts. The third line is an example of the command. 3. The purpose of the RegisterAppClass function is to register window class templates that are used by the GDIDEMO application to create windows. Use the CLASS command to examine the classes registered by GDIDEMO. :CLASS GDIDEMO Note: This example shows only those classes specifically registered by the GDIDEMO application. Classes registered by other Windows modules, such as USER32, are omitted. The output of the CLASS command provides summary information for each window class registered on behalf of the GDIDEMO process. This includes the class name, the address of the internal WINCLASS data structure, the module which registered the class, the address of the default window procedure for the class, and the value of the class style flags. Note: For more specific information on window class definitions, use the CLASS command with the -X option, as follows: :CLASS -X Class Name Handle Owner WndwProc Styles ---------------Application Private--------------- BOUNCEDEMO A018A3B0 GDIDEMO 004015A4 00000003 DRAWDEMO A018A318 GDIDEMO 00403CE4 00000003 MAZEDEMO A018A280 GDIDEMO 00403A94 00000003 XFORMDEMO A018A1E8 GDIDEMO 00403764 00000003 POLYDEMO A018A150 GDIDEMO 00402F34 00000003 GDIDEMO A018A0C0 GDIDEMO 004010B5 00000003 Using Symbols and Symbol Tables Now that you are familiar with using SoftICE to step, trace, and create point-and-shoot style breakpoints, it is time to explore symbols and tables. When you load symbols for an application, SoftICE creates a symbol table that contains all the symbols defined for that module. 1. Use the TABLE command to see all the symbol tables that are loaded: :TABLE GDIDEMO [NM32] 964657 Bytes Of Symbol Memory Available The currently active symbol table is listed in bold. This is the symbol table used to resolve symbol names. If the current table is not the table from which you want to reference symbols, use the TABLE command and specify the name of the table to make active: :TABLE GDIDEMO 2. Use the SYM command to display the symbols from the current symbol table. With the current table set to GDIDEMO, the SYM command produces output similar to the following abbreviated output: :SYM .text(001B) 001B:00401000 WinMain 001B:004010B5 WndProc 001B:004011DB CreateProc 001B:00401270 CommandProc 001B:00401496 PaintProc 001B:004014D2 DestroyProc 001B:004014EA lRandom 001B:00401530 CreateBounceWindow 001B:004015A4 BounceProc 001B:004016A6 BounceCreateProc 001B:00401787 BounceCommandProc 001B:0040179C BouncePaintProc This list of symbol names is from the .text section of the executable. The .text section is typically used for procedures and functions. The symbols displayed in this example are all functions of GDIDEMO. Setting a Conditional Breakpoint One of the symbols defined for the GDIDEMO application is the LockWindowInfo function. The purpose of this routine is to retrieve a pointer value that is specific to a particular instance of a window. To learn about conditional and memory breakpoints, you will perform the following steps: * Set a BPX breakpoint on the LockWindowInfo function. * Edit the breakpoint to use a conditional expression, thus setting a conditional breakpoint. * Set a memory breakpoint to monitor access to a key piece of information, as described in Setting a Read-Write Memory Breakpoint on page 39. Setting a BPX Breakpoint Before setting the conditional breakpoint, you need to set a BPX-style breakpoint on LockWindowInfo. 1. Set a BPX-style breakpoint on the LockWindowInfo function: :BPX LockWindowInfo When one of the GDIDEMO windows needs to draw information in its client area, it calls the LockWindowInfo function. Every time the LockWindowInfo function is called, SoftICE pops up to let you debug the function. The GDIDEMO windows continually updates, so this breakpoint goes off quite frequently. 2. Use the BL command to verify that the breakpoint is set. 3. Use either the X or G command to exit SoftICE. SoftICE should pop up almost immediately on the LockWindowInfo function. Editing a Breakpoint From the LockWindowInfo function prototype on source line 47, you can see that the function accepts one parameter of type HWND and returns a void pointer type. The HWND parameter is the handle to the window that is attempting to draw information within its client area. At this point, you want to modify the existing breakpoint, adding a conditional breakpoint to isolate a specific HWND value. 1. Before you can set the conditional expression, you need to obtain the HWND value for the POLYDEMO window. The HWND command provides information about application windows. Use the HWND command and specify the GDIDEMO process: :HWND GDIDEMO The following example illustrates what you should see if you are using Windows NT. If you are using Windows 95, your output will vary. Handle Class WinProc TID Module 07019C GDIDEMO 004010B5 2D GDIDEMO 100160 MDIClient 77E7F2F5 2D GDIDEMO 09017E BOUNCEDEMO 004015A4 2D GDIDEMO 100172 POLYDEMO 00402F34 2D GDIDEMO 11015C DRAWDEMO 00403CE4 2D GDIDEMO The POLYDEMO window handle is bold and underlined. This is the window handle you want to use to form a conditional expression. If the POLYDEMO window does not appear in the HWND output, exit SoftICE using the G or X commands (F5) and repeat Step 1 until the window is created. The value used in this example is probably not the same value that appears in your output. For the exercise to work correctly, you must use the HWND command to obtain the actual HWND value on your system. Using the POLYDEMO window handle, you can set a conditional expression to monitor calls to LockWindowInfo looking for a matching handle value. When the LockWindowInfo function is called with the POLYDEMO window handle, SoftICE pops up. 2. Because you already have a breakpoint set on LockWindowInfo, use the BPE command (Breakpoint Edit) to modify the existing breakpoint: :BPE 0 When you use the BPE command to modify an existing breakpoint, SoftICE places the definition of that breakpoint onto the command line so that it can be easily edited. The output of the BPE command appears: :BPX LockWindowInfo The cursor appears at the end of the command line and is ready for you to type in the conditional expression. 3. Remember to substitute the POLYDEMO window handle value that you found using the HWND command instead of the value (100172) used in this example. Your conditional expression should appear similar to the following example. The conditional expression appears in bold type. :BPX LockWindowInfo IF ESP->4 == 100172 Note: Win32 applications pass parameters on the stack and at the entry point of a function; the first parameter has a positive offset of 4 from the ESP register. Using the SoftICE expression evaluator, this is expressed in the following form: ESP->4. ESP is the CPU stack pointer register and the "->" operator causes the lefthand side of the expression (ESP) to be indirected at the offset specified on the righthand side of the expression (4). For more information on the SoftICE expression evaluator refer to Chapter 8: Using Expressions on page 125 and for referencing the stack in conditional expressions refer to Conditional Breakpoints on page 114. 4. Verify that the breakpoint and conditional expression are correctly set by using the BL command. 5. Exit SoftICE using the G or X command (F5). When SoftICE pops up, the conditional expression will be TRUE. Setting a Read-Write Memory Breakpoint We set the original breakpoint and subsequently the conditional expression so that we could obtain the address of a data structure specific to this instance of the POLYDEMO window. This value is stored in the windowÆs extra data and is a global handle. The LockWindowInfo function retrieves this global handle and uses the Win32 API LocalLock to translate it into a pointer that can be used to access the windowÆs instance data. 1. Obtain the pointer value for the windows instance data by executing up to the return statement on source line 57: :G .57 2. Win32 API functions return 32-bit values in the EAX register, so you can use the BPMD command and specify the EAX register to set a memory breakpoint on the instance data pointer. :BPMD EAX The BPMD command uses the hardware debug registers provided by Intel CPUs to monitor reads and writes to the Dword value at a linear address. In this case, you are using BPMD to trap read and write accesses to the first Dword of the window instance data. 3. Use the BL command to verify that the memory breakpoint is set. Your output should look similar to the following: :BL 00) BPX LockWindowInfo IF ((ESP->4)==0x100172) 01) BPMD #0023:001421F8 RW DR3 Breakpoint index 0 is the execution breakpoint on LockWindowInfo and breakpoint index 1 is the BPMD on the window instance data. 4. Use the BD command to disable the breakpoint on the LockWindowInfo. :BD 0 SoftICE provides the BC (breakpoint clear) and BD (breakpoint disable) commands to clear or disable a breakpoint. Disabling a breakpoint is useful if you want to re-enable the breakpoint later in your debugging session. If you are not interested in using the breakpoint again, then it makes more sense to clear it. 5. Use the BL command to verify that the breakpoint on LockWindowInfo is disabled. SoftICE indicates that a breakpoint is disabled by placing an asterisk (*) after the breakpoint index. Your output should appear similar to the following: :BL 00) * BPX _LockWindowInfo IF ((ESP->4)==0x100172) 01) BPMD #0023:001421F8 RW DR3 Note: You can use the BE command to re-enable a breakpoint: :BE breakpoint-index-number 6. Exit SoftICE using the G or X command. When the POLYDEMO window accesses the first Dword of its window instance data, the breakpoint triggers and SoftICE pops up. When SoftICE pops up due to the memory breakpoint, you are in the PolyRedraw or PolyDrawBez function. Both functions access the nBezTotal field at offset 0 of the POLYDRAW window instance data. Note: The Intel CPU architecture defines memory breakpoints as traps, which means that the breakpoint triggers after the memory has been accessed. In SoftICE, the instruction or source line that is highlighted is the one after the instruction or source line that accessed the memory. 7. Clear the breakpoints you set in this section by using the BC command: :BC * Note: You can use the wildcard character (*) with the BC, BD, and BE commands to clear, disable, and enable all breakpoints. 8. Exit SoftICE using the G or X command. The operating system terminates the application. Congratulations on completing your first SoftICE debugging session. In this session, you traced through source code, viewed locals and structures, and set point-and-shoot, conditional, and read-write memory breakpoints. SoftICE provides many more advanced features. The SoftICE commands ADDR, HEAP, LOCALS, QUERY, THREAD, TYPES, WATCH, and WHAT are just a few of the many SoftICE commands that help you debug smarter and faster. Refer to the SoftICE Command Reference for a complete explanation of all the SoftICE commands.